On the BICM Capacity 



1 



Erik Agrell and Alex Alvarado 
Department of Signals and Systems, Communication Systems Group 
Chalmers University of Technology, Gothenburg, Sweden 
{agrell,alex.alvarado]@ chalmers.se 



Abstract 

Optimal binary labelings, input distributions, and input alphabets are analyzed for the so-called 
bit-interleaved coded modulation (BICM) capacity, paying special attention to the low signal-to-noise 
ratio (SNR) regime. For 8-ary pulse amplitude modulation (PAM) and for 0.75 bit/symbol, the folded 
binary code results in a higher capacity than the binary reflected gray code (BRGC) and the natural 
binary code (NBC). The 1 dB gap between the additive white Gaussian noise (AWGN) capacity and 
the BICM capacity with the BRGC can be almost completely removed if the input symbol distribution 
is properly selected. First-order asymptotics of the BICM capacity for arbitrary input alphabets and 
distributions, dimensions, mean, variance, and binary labeling are developed. These asymptotics are 
used to define first-order optimal (FOO) constellations for BICM, i.e., constellations that make BICM 
achieve the Shannon limit —1.59 dB. It is shown that the E^/No required for reliable transmission at 
asymptotically low rates in BICM can be as high as infinity, that for uniform input distributions and 
8-PAM there are only 72 classes of binary labelings with a different first-order asymptotic behavior, 
and that this number is reduced to only 26 for 8-ary phase shift keying (PSK). A general answer to the 
question of FOO constellations for BICM is also given: using the Hadamard transform, it is found that 
for uniform input distributions, a constellation for BICM is FOO if and only if it is a linear projection 
of a hypercube. A constellation based on PAM or quadrature amplitude modulation input alphabets is 
FOO if and only if they are labeled by the NBC; if the constellation is based on PSK input alphabets 
instead, it can never be FOO if the input alphabet has more than four points, regardless of the labeling. 
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I. Introduction 

The problem of reliable transmission of digital information through a noisy channel dates 
back to the works of Nyquist JT), and Hartley j3]| almost 90 years ago. Their efforts were 
capitalized by C. E. Shannon who formulated a unified mathematical theory of communication in 
1948 flU, ll5lrl After he introduced the famous capacity formula for the additive white Gaussian 
noise (AWGN) channel, the problem of designing a system that operates close to that limit 
has been one of the most important and challenging problems in information/communication 
theory. While low spectral efficiencies can be obtained by combining binary signaling and a 
channel encoder, high spectral efficiencies are usually obtained by using a coded modulation 
(CM) scheme based on a multilevel modulator. 

In 1974, Massey proposed the idea of jointly designing the channel encoder and modulator 0, 
which inspired Ungerboeck's trellis-coded modulation (TCM) (HI, flU and Imai and Hirakawa's 
multilevel coding (MLC) OH, flTT). Since both TCM and MLC aim to maximize a Euclidean 
distance measure, they perform very well over the AWGN channel. However, their performance 
over fading channels is rather poor. The next breakthrough came in 1992, when Zehavi introduced 
the so-called bit-interleaved coded modulation (BICM) [12] (later analyzed in [13]), which is a 
serial concatenation of a binary channel encoder, a bit-level interleaver, and a memory less mapper. 
BICM aims to increase the code diversity — the key performance measure in fading channels — 
and therefore outperforms TCM in this scenario lfT3l Table III]. BICM is very attractive from an 
implementation point view because of its flexibility, i.e., the channel encoder and the modulator 
can be selected independently, somehow breaking Massey's joint design paradigm. BICM is 
nowadays a de facto standard, and it is used in most of the existing wireless systems, e.g., HSPA 
(HSDPA and HSUPA) El ES Ch. 12], IEEE 802.1 la/g 0U IEEE 802.1 In El Sec. 20.3.3], 
and the latest DVB standards (DVB-T2 QJ|, DVB-S2 030, and DVB-C2 11201 ). 

Plots of the BICM capacity vs. E h /N reveal that BICM does not always achieve the Shannon 
limit (SL) —1.59 dB. This can be explained based on first-order asymptotics of the BICM 
capacity, which were recently developed by Martinez et al. for uniform input distributions and 
one- and two-dimensional input alphabets lETI . 11221 . It was shown that there is a bounded 
loss between the BICM capacity and the SL when pulse amplitude modulation (PAM) input 

'An excellent summary of the contributions that influenced Shannon's work can be found in ||6] Sec. I]. 
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alphabets labeled by the binary reflected gray code (BRGC) is used. Recently, Stierstorfer and 
Fischer showed in ll23l that this is caused by the selection of the binary labeling and that equally 
spaced PAM and quadrature amplitude modulation (QAM) input alphabets with uniform input 
distributions labeled by the natural binary code (NBC) achieve the SL. Moreover, the same 
authors showed in ll24l that for low to medium signal-to-noise ratios (SNR), the NBC results 
in a higher capacity than the BRGC for PAM and QAM input alphabets and uniform input 
distributions. 

The fact that the BICM capacity does not always achieve the SL raises the fundamental 
question about first-order optimal (FOO) constellations for BICM, i.e., constellations that make 
the BICM achieve the SL. In this paper, we generalize the first-order asymptotics of the BICM 
capacity presented in ||2U| to input alphabets with arbitrary dimensions, input distributions, mean, 
variance, and binary labelings. Based on this model, we present asymptotic results for PAM and 
phase shift keying (PSK) input alphabets with uniform input distribution and different binary 
labelings. Our analysis is based on the so-called Hadamard transform ll25l pp. 53-54], which 
allows us to fully characterize FOO constellations for BICM with uniform input distributions for 
fading and nonfading channels. A complete answer to the question about FOO constellations for 
BICM with uniform input distributions is given: a constellation is FOO if and only if it is a linear 
projection of a hypercube. Furthermore, binary labelings for the traditional input alphabets PAM, 
QAM, and PSK are studied. In particular, it is proven that for PAM and QAM input alphabets, 
the NBC is the only binary labeling that results in an FOO constellation. It is also proven that 
PSK input alphabets with more than four points can never yield an FOO constellation, regardless 
of the binary labeling. When 8-PAM with a uniform input distribution is considered, the folded 
binary code (FBC) results in a higher capacity than the BRGC and the NBC. Moreover, it is 
shown how the BICM capacity can be increased by properly selecting the input distribution, 
i.e., by using so-called probabilistic shaping ll26l . In particular, probabilistic shaping is used to 
show that PAM input alphabets labeled by the BRGC or the FBC can also be FOO, and to show 
that the 1 dB gap between the AWGN capacity and the BICM capacity with the BRGC can be 
almost completely removed. 
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II. Preliminaries 

A. Notation Convention 

Hereafter we use lowercase letters x to denote a scalar, boldface letters x to denote a row 
vector of scalars, and underlined symbols x to denote a sequence. Blackboard bold letters X 
represent matrices and Xij represents the entry of X at row i, column j, where all the indices 
start at zero. The transpose of X is denoted by X T , trace (X) is the trace of X, and ||X|| 2 is 
trace (X T X) . 

We denote random variables by capital letters Y, probabilities by Pr{-}, the probability 
mass function (pmf) of the random vector Y by P Y (y), and the probability density function 
(pdf) of the random vector Y by Py{v)- The joint pdf of the random vectors X and Y 
is denoted by px,Y{x,y), an d the conditional pdf of Y conditioned on X = x is denoted 
by Py\x=x(u)- The same notation applies to joint and conditional pmfs, i.e., P X x{ x ^v) an d 
Py\x=x(u)- The expectation of an arbitrary function f(X, Y) over the joint pdf of X and Y 
is denoted by Ex,y[/(X,F)], the expectation over the conditional pdf Py\x=x(v) is denoted 
by ~E Y \x=x[f(X, Y)], and cov (X) is the covariance matrix of the random vector X. 

We denote the base-2 representation of the integer < % < M — 1, where M = 2 m , by the 
vector b(i) = [b m -i(i), b m -2(i), ■ ■ ■ , bo(i)], where 6 m _i(z) is the most significant bit of i and 
b (i) the least significant. To facilitate some of the developments in this paper, we also define 
the ordered direct product as 

[al . . . , aJ_J T ® [bl b^f = [cl . . . , cj^p, (1) 

where c qi+ j = [oj, bj] for i — 0, . . . ,p — 1 and j — 0, . . . , q — 1. The ordered direct product in 
<0Q) is analogous to the Cartesian product except that it operates on vectors/matrices instead of 
sets. 

B. Binary Labelings 

A binary labeling L of order m > 1 is defined using an M x m matrix where each row 
corresponds to one of the M length-m distinct binary codewords, L = [cj , . . . , c^ / _ 1 ] T , where 

Ci = [ci j0 , Ci,i, . . . , c i>m _i] e {0, i} m . 
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In order to recursively define some particular binary labelings, we first define expansions, repe- 
titions, and reflections of binary labelings. To expand a labeling L m = [cq , . . . , c^ f _ 1 ] T into a la- 
beling L m+ i, we repeat each binary codeword once to obtain a new matrix [cj , Cq , . . . , c^ f _ 1 , cJ / _ 1 ] T , 
and then we obtain L m+1 by appending one extra column [0, 1, 1, 0, 0, 1, 1, 0, . . . , 0, 1, 1, 0] T 
of length 2M E71 . To generate a labeling L m+1 from a labeling L m = [cj , . . . , cJ / _ 1 ] T by 
repetition, we repeat the labeling L m once to obtain a new matrix [cq , . . . , c| ? _ 1 , , . . . , c| f _ 1 ] T , 
and we add an extra column from the left, consisting of M zeros followed by M ones. Finally, 
to generate a labeling L m+1 from a labeling L m = [cq , . . . , c^ f _ 1 ] T by reflection, we join L m 
and a reversed version of L m to obtain a new matrix [cj , . . . , c^_ 1; c^_ 1; . . . , ] t , and we 
add an extra column from the left, consisting of M zeros followed by M ones [|27l . 

In this paper we are particularly interested in the binary reflected Gray code (BRGC) [|28l . 
E9l , the natural binary code (NBC), and the folded binary code (FBC) ll30ll . The FBC was 
analyzed in 113011 for uncoded transmission and here we will, to our knowledge for the first time, 
consider it for coded transmission. In Sec. IIII-DI and Sec. IV-CI it is shown to yield a higher 
capacity than other labelings under some conditions. We also introduce a new binary labeling 
denoted binary semi-Gray code (BSGC). These binary labelings are generated as follows: 

• The BRGC G m of order m > 1 is generated by m — 1 recursive expansions of the trivial 
labeling Li = [0, 1] T , or, alternatively, by m — 1 recursive reflections of Li. 

• The NBC N m of order m > 1 is defined as the codewords q that are the base-2 represen- 
tations of the integers i = 0, . . . , M - 1, i.e., N m = [6(0) T , . . . , b(M - 1) T ] T . Alternatively, 
N m can be generated by m — 1 recursive repetitions of the trivial labeling Li, or as m — 1 
ordered direct products of Li with itself. 

• The BSGC S m of order m > 3 is generated by replacing the first column of G m by the 
modulo-2 sum of the first and last columns. 

• The FBC F m of order m > 2 is generated by one reflection of N m _i. 

For any labeling matrix L = [<$ , . . . , cJ f _ 1 ] T , where Cj = [c i>0 , . . . , Q, m _i] G {0, l} m , we 
define a modified labeling matrix Q = Q(L) which is obtained by reversing the order of the 
columns and applying the mapping (0 — > 1, 1 — > —1), i.e., 

T, if (k,m-l-k = 1 

Qi,k = \ (2) 

"1? if Cj.m-l-fe = 
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with i = 0, . . . , M - 1 and k = 0, . . . , m - 1. 
Example 1 (Binary labelings of order m = 3): 
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C. Constellations and Input Distributions 

Throughout this paper, we use X to represent the set of symbols used for transmission. Each 
element of X is an iV-dimensional symbol cCj, % — 0, . . . , M — 1, where \X\ = M = 2 m and 
X C R^. We define the input alphabet using an M x m matrix X = [xq, . . . , a?jyi-i] T which 
contains all the elements of X. 

For practical reasons, we are interested in well structured input alphabets. An M-PAM input 
alphabet is defined by the column vector X P am where x^i = — (M— 2i — 1) with i = 0, . . . , M— 1. 



An Af-PSK input alphabet is the matrix Xpsk where xi 



cos 



(2i + l)7T 

M 



, sin 



(2i + l)7T 

M 



with i 



0,1, 



M - 1. Finally, a rectangular (M' x M")-QAM input alphabet is the M'M" x 2 matrix 
Xqam = Xp AM <g) Xp AM , where X PAM and Xp AM are vectors of length M' and M", respectively. 
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For a given input alphabet X, the input distribution of the symbols is denoted by the pmf 
Px(x), which represents the probabilities of transmitting the symbols x, i.e., Pr{X = x}. We 
define the matrix P as an ordered list containing the probabilities of the symbols, i.e., P = 
[P x (x ), -PxOa/-i)] T - We use U M = [1/M, . . . , 1/M] T to denote the discrete uniform 
input distribution. 

We define a constellation as the list of matrices f2 = [X, L, P], i.e., an input alphabet using 
a given labeling and input distribution. Finally, for a given pair [X, L], we denote with X^ u C 
{0, . . . , M — 1} the set of indexes of the symbols with a binary label u G {0, 1} at bit position 

k G {0, . . . , m - 1}, i.e., I k>u = {i G {0, . . . , M - 1} : Q, fc = u}. 

D. System Model 

In this paper, we analyze coded modulation schemes (CM) as the one shown in Fig.Q] Each of 
the K possible messages is represented by the binary vector w G {0, l} fcc , where k c = log 2 K. 
The transmitter maps each message to a sequence x = [x(0) T , . . . , x(N s — 1) T ] T G X Ns , which 
corresponds to N s A^-dimensional symbols (N s channel useo. The code C is a subset of X Ns 
such that |C| = K, which is used for transmission. The transmitter is then defined as a one-to-one 

function that assigns each information message w to one of the K possible sequences x G C 

k c 

The code rate in information bits per coded bits is then given by R = or, equivalently, 

mJV s 

R c = — information bits per channel use (information bits per symbol, or information bits 
per N real dimensions). At the receiver's side, based on the channel observations, a maximum 
likelihood sequence receiver generates an estimate of the information bits w selecting the most 
likely transmitted message. 

We consider transmissions over a discrete-time memoryless fast fading channel 

Y(t) = H(t)oX{t) + Z{t), (3) 

where the operator o denotes the so-called Schur product (element-wise product) between two 
vectors, X(t), H(t), Y(t), and Z(t) are the underlying random vectors for x{t), h{t), y(t), and 
z(t) respectively, with t = 0, . . . , N s — 1 being the discrete time index, and Z(t) is a Gaussian 

2 A "channel use" corresponds to the transmission of one iV-dimensional symbol, i.e., it can be considered as a "vectorial 
channel use". 
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BICM Channel 




Figure 1. A CM system based on a BICM structure: A binary channel encoder, a bit-level interleaver, a memoryless mapper, 
the fading channel, and the inverse processes at the receiver side. 



noise with zero mean and variance N /2 in each dimension. The channel is represented by the 

^-dimensional vector H(t), and it contains real fading coefficients Hi which are assumed to be 

random variables, possibly dependent, with same pdf pu{h). We assume that H(t) and N are 

perfectly known at the receiver or can be perfectly estimated. Since the channel is memoryless, 

from now on we drop the index t. 

The conditional transition pdf of the channel in ([3]) is given by 

. , 1 / \\y — h o x\\ 2 \ 
PYl x=„, H = h (y) = j^j, exp ^ j . (4) 

We assume that both H and X have finite and nonzero second moments, that X, H, and Z 
are mutually independent, and that there exists a constant u > such that for all sufficiently 
large A > the vector H satisfies 

Pt{\\H\\ 2 > A} < exp(-A aJ ). (5) 

This condition will be used in the proof of Theorem [7] in Sec. IIV-CI 

Each transmitted symbol conveys R c information bits and thus, the relation between the 
average symbol energy E s = Ex[||^|| 2 ] and the average information bit energy is given by 
E s = R c E h . We define the average signal-to-noise ratio (SNR) as 

SNR = VHAWHoXf] = Eh[h2] E^ = Eh[H 2 ]Rc ^ (6) 

iV iv iV 

The AWGN channel is obtained as a special case of © by taking H as the all-one vector. An- 
other particular case is obtained when H — H\ — . . . — -f/jv-i = A, which particularizes to the 
Rayleigh fading channel when A = \J A\ + A 2 and Ai, A 2 are independent zero-mean Gaussian 
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random variables. In this case, the instantaneous SNR defined by Ex[||iT°-X'|| 2 ]/iVo = A 2 E S /N 
follows a chi-square distribution with one degree of freedom (an exponential distribution). Sim- 
ilarly, the Nakagami-m fading channel is obtained when A follows a Nakagami-m distribution. 
It can be shown that the condition © is fulfilled in all the cases above. 

In a BICM system |fl2l . |[T3l . the transmitter in Fig. [T] is realized using a serial concatenation 
of a binary encoder of rate R = R c /m, a bit level interleaver, and a memory less mapper $. 
The mapper $ is defined as a one-to-one mapping rule that maps the length-m binary random 
vector C = [C , . . . , C m _i] to one symbol X, i.e., $ : {0, l} m — > X. At the receiver's side, the 
demapper computes soft information on the coded bits, which are then deinterleaved and passed 
to the channel decoder. The a posteriori L-values for the A;th bit in the symbol and for a given 
fading realization are given by 

7 / \ a I Pr{Y = y\C k = l,H = h} 

hiy) = bgc Pr{Y = y\C k = 0,H = h} (7) 
= E (-D U+1 ^e E «P ( Jy ~ h N ° Xir ): (8) 

E ( _1 ) n min \\y- hox i\\ 2 , (9) 

iVo ^ , i&X k . u 
«G{0,1} 

where to pass from ([8]) to ©, the so-called max-log OTTl approximation was used. 

The max-log metric in © (already proposed in lfT2ll . |fT3l0 is suboptimal; however, it is very 
popular in practical implementations because of its low complexity, e.g., in the 3rd generation 
partnership project (3GPP) working groups ll32ll . It is also known that when Gray-labeled 
constellations are used, the use of this simplification results in a negligible impact on the 
receiver's performance ll33l Fig. 9] ll34l Fig. 6]. The max-log approximation also allows BICM 
implementations which do not require the knowledge of iV , for example, when a Viterbi decoder 
is used, or when the demapper passes hard decisions to the decoder. Moreover, the use of the max- 
log approximation transforms the nonlinear relation lk(y) in © into a piecewise linear relation. 
This has been used to develop expressions for the pdf of the L-values in © using arbitrary 
input alphabets ll35l (based on an algorithmic approach), closed-form expressions for QAM 
input alphabets labeled by the BRGC for the AWGN channel (361, J33, and for fading channels 
[|38l . Recently, closed-form approximations for the pdf of the L-values in © for arbitrary input 
alphabets and binary labeling in fading channels have been presented lf3~9l . 
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E. The Hadamard Transform 

The Hadamard transform (HT) is a discrete, linear, orthogonal transform, like for example the 
Fourier transform, but its coefficients take values in ±1 only. Among the different applications 
that the HT has, one that is often overlooked is as an analysis tool for binary labeling s 11401 . 
|]4T1|. The HT is defined by means of an M x M matrix, the Hadamard matrix, which is defined 
recursively as follows when M is a power of two (251 PP- 53-54]. 

Hm Hm 
Hm — Hjif 



Hi = 1 



Example 2 (Hadamard matrix 
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(10) 



In the following, we will drop the index, letting H represent a Hadamard matrix of any size 
M = 2 m . Hadamard matrices have the following appealing properties. 



H = EL 



M 



-EL 



(11) 



It can be shown H2l Sec. 1.1] |[43l Sec. Ill] that the elements of a Hadamard matrix are h 
[Yk=o (~^) bk ^ bk ^ ' from which we observe for future use that for all i — 0, . 
I — 0, . . . , m — 1, 



M - 1 and 



m—l 



h 



L0 



hi 



2' 



IK- 1 ) 



_l)bl(i) 



(12) 



fc=0 



where bi(i) is the /th bit of the base-2 representation of the integer i. 

At this point it is interesting to note the close relation between the columns of the matrix 
Q(N 3 ) in Example \T\ and the columns 2 l of H 8 in (flOl) for I = 0, 1, 2. Its generalization is given 
by the following lemma, whose proof follows immediately from ©, the definition of the NBC 
in Sec. HE and CEI]). 
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Lemma 1: Let Q = Q(N m ) be the modified labeling matrix for the NBC of order m, and let 
EI be the Hadamard matrix. For any m, and for k — 0, . . . , m — 1 and i — 0, . . . , M — 1, 

Qi,k = h { ,2 k - (13) 

The HT operates on a vector of length M = 2 m , for any integer m, or in a more general case, 
on a matrix with M = 2 m rows. The transform of a matrix X is denoted X and has the same 
dimensions as X. It is defined as 

x = —ex (14) 

M V ; 

and the inverse transform is X = HX. Equivalently, 

^ M-l M-l 
i=0 j=0 

where from (fTTI) we have that = /ijj, and where we have introduced the row vectors and 
Xj such that 

X = |_a? , • • • , *M-lJ J ^ = [^0 ) • • • ) X M-l\ ■ 

Because of (fT2l) . the first element of the transform is simply x = j-j X]f=o 1 x %- 



vM-: 



Finally, using J2j=o \\ x j\\ 2 = trace (X T X), (fl4l) . and (fTTI) . we note that a variant of Parseval's 
theorem holds: 

M-l M-l 
j=0 i=0 

III. Capacity of Coded Modulation Systems 

In this section we analyze the capacity of CM schemes, i.e., the so-called CM and BICM 
capacities. We review their relation and we analyze how the selection of the constellation 
influences them. We pay special attention to the selection of the binary labeling and the use 
of probabilistic shaping for BICM. 
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A. AMI and Channel Capacity 

In this subsection, we assume the use of a continuous input alphabet, i.e., X = ~R N , which 
upperbounds the performance of finite input alphabets. 

The average mutual information (AMI) in bits^ per channel use between the random vectors 
X and Y when the channel is perfectly known at the receiver is defined as 

Pxx (X,Y) 



I X (X;Y)^V XX 



E 



X.Y 



log, 
log 2 



2 p Y {Y)p x {X) 
Py\x(Y) 
MY) 



(17) 
(18) 



where we use X as the index of I X (X; Y) to emphasize the fact that the AMI depends on the 
input PDF p x (x). For an arbitrary channel parameter H, the AMI in (flTl) can be expressed as_ 

Py\x,h{Y 



Ix(X; Y) = Exy,f/ 



log 2 



Py\h( 



(19) 



where p Y \x= x ,H=h{y) is given by ©. 

The channel capacity of a continuous-input continuous-output memoryless channel is defined 
as the maximum AMI between its input and output p4l Ch. 4] B51 eq. (3)] 



C(SNR) = nmx I X {X;Y) 

p x (x) 



(20) 



where the maximization is over all possible input distributions. The capacity in (1201) has units 
of [bit/channel use] (or equivalently [bit/symbol]), and it is an upper bound on the number of 
bits per symbol that can be reliably transmitted through the channel, where a symbol consists of 
N real dimensions. Shannon's channel coding theorem states that it is not possible to transmit 
information reliably above this fundamental limit, i.e., 



J R C <C(SNR) = C( J R C E H [// 2 ]^ 



(21) 



The AWGN capacity, denoted by C AW (SNR), is defined as the channel capacity of the AWGN 
channel (obtained from © using H(t) = 1), and it is given by [|44l Sec. 9.4] 



C Aw (SNR) = | log2 (l + ^SNR 



(22) 



throughout this paper all the AMIs are given in bits. 

4 We note that the AMI with perfect channel state information is usually denoted by Ix{X; Y\H), however, and for notation 



simplicity, we use Ix(X; Y). 
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This capacity is attained when X are i.i.d. zero-mean Gaussian random variables with variance 
E s /N in each dimension and it follows from the fact that the noise is independent in each 
dimension, and thus, the transmission of X can be considered as a transmission through N 
parallel independent Gaussian channels. 

We define the conditional AMI for discrete input alphabets as the AMI between X and Y 
conditioned on the outcome of a third random variable U, i.e., 

Py\x,u=u(Y 



Ix\u=u(X; Y) — ~Ex,y\u=u 



log 



E 



X,Y,H\U=u 



PY\U=u(Y) 
PY\X,H,U=u(Y) 



log 



PY\H,U=u\ 



(23) 
(24) 



which is valid for any random H. 



B. CM Capacity 

The CM capacity is defined as the AMI between X and Y for a given constellation fi, i.e., 

I™(SNR) ±I X (X;Y) (25) 
= Ix(C-Y) (26) 

m— 1 

= Ys I x(C k ;Y\C ,...,C k _ 1 ), (27) 

fc=0 

where to pass from (1251) to (|26|) . we used the fact that the mapping rule between C and X 
is one-to-one. To pass from (1261) to (1271) we have used the chain rule of mutual information 
H4l Sec. 2.5], where Ix(C k ; Y\C , . . . , Ck-i) represents a bit level AMI which represents the 
maximum rate that can be used at the [k + l)th bit position, given a perfect knowledge of the 
previous k bits. 

The CM capacity in (1251) corresponds to the capacity of the memoryless "CM channel" in Fig.Q] 
for a given constellation fl We note that different binary labelings will produce different values 
of Ix{Ck] Y\Cq, . . . , Cfe_i) in (l27l) ; however, the overall sum will remain constant, i.e., the CM 
capacity does not depend on the binary labeling. We use the name "CM capacity" for I™ (SNR) 
in (g5]) following the standard terminology^] used in the literature (cf. ifBll. lUTI. 11231. |gg|. B9l). 
although we recognize the misusage of the word capacity since no optimization over the input 

5 Sometimes, this is also called joint capacity 1261 . or (constellation) constrained capacity 1461 . 1471 . 
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distribution is performed (cf. (1201)). Moreover, it is also possible to optimize the input alphabet in 
order to obtain an increase in the AMI (so-called signal shaping ll50l ). Nevertheless, throughout 
this paper we will refer to the AMI for a given f2 in (|25l ) as the CM capacity. 

In this paper we are interested in optimal constellations, and therefore, we define the maximum 
CM capacity as 

C CM (SNR) 4 max l£ M (SNR) (28) 

m— 1 

= max V I x (C k ; Y\C , C fc _i). (29) 

fe=0 

As mentioned before, the CM capacity does not depend on the binary labeling, i.e., it does not 
depend on how the mapping rule $ is implemented, and therefore, in (|29l we only show two 
optimization parameters: the input alphabet and the input distribution. 

The CM capacity in (1251) (for a given constellation Vt) is an upper bound on the num- 
ber of bits per symbol that can be reliably transmitted using for example TCM [0 or MLC 
with multistage decoding (MLC-MSD) JlOl, ED- MLC-MSD is in fact a direct application 
of the summation in (|27T) . i.e., m parallel encoders are used, each of them having a rate 
R k = Ix{Ck', Y\C , . . . , Cfc_i). At the receiver's side, the first bit level is decoded and the 
decisions are passed to the second decoder, which then passes the decisions to the third decoder, 
and so on. Other design rules can also be applied in MLC, cf. [|5TI . The maximum CM capacity 
C CM (SNR) in (|29l) represents an upper bound on the number of bits per symbol that can be 
reliably transmitted using a fully optimized system, i.e., a system where for each SNR value 
SNR, the input alphabet and the input distribution are selected in order to maximize the CM 
capacity I™ (SNR). 

C. BICM with Arbitrary Input Distributions 

It is commonly assumed that the sequence generated by the binary encoder in Fig.CQis infinitely 
long and symmetric, and also that the interleaver (ir) operates over this infinite sequence, simply 
permuting it in a random way. Under these standard assumptions, it follows that the input symbol 
distribution will be always P = Um- Since in this paper we are interested in analyzing a more 
general setup where the input symbol distribution can by modified, we develop a more general 
model in which we relax the equiprobable input distribution assumption. 
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Let Ck G {0, 1} the binary random variable representing the bits at the kth modulator's 
input, where the pmf Pc k (u) represents the probability of transmitting a bit u at bit position 
k. We assume that in general J2T=o ^c k (^) 7^ Y^T=o i- e -> me coded and interleaved 

sequence could have more zeros than ones (or vice- versa). Note that since Po k ( u ) is a P m f> 

P Cfc (0) + P Cfc (l) = l. 

Let Cj = [c ij0 , • • • , Cj,m-i] be the binary label of the symbol cCj. We assume that the bits at the 
input of the modulator are independent, and therefore, the input symbol probabilities are 



m—l 



P x (x t )=]lP Ck (c hk ). 



(30) 



A;=0 



The independence condition on the coded bits that results in (1301) can be obtained if the 
interleaver block in Fig. \T\ completely breaks the temporal correlation of the coded bits. The 
condition that the coded and interleaved sequence could be asymmetric can be obtained for 
example by using an encoder with nonuniform outputs, or by a particular puncturing scheme 
applied to the coded bits. This can be combined with the use of multiple interleavers and 
multiplexing ll52~ll . which would allow Pc k (u) ^ 1/2. Examples of how to construct a BICM 
scheme where nonuniform input symbol distributions are obtained include the "shaping encoder" 
of 11531 . Il54ll and the nonuniform signaling scheme based on a Huffman code of [|55l . 

For future use, we also define the conditional input symbol probabilities, conditioned on the 
A;th bit being u, as 



Px\C k =u(Xi) 



( m-1 

[ Pc k ,( c i,k'), 



if Ci 



i.k 



U 



k'=0 
k'^k 

0. 

Pc k (u) 
0. 



if Cj jfe ^ u 

if i E l k , u 
if i £ 2 k>u 



(3D 



where X, % is defined in Sec. III-CI 



D. BICM Capacity 

The "BICM channel" in Fig. \T\ was introduced in Il56l and it is what separates the encoder 
and decoder in a BICM system. The BICM capacity is then defined as the capacity of the BICM 
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channel. Using the definitions in Sec. IIII-CI and the equivalent channel model in ||T3l Fig. 6], 
which replaces the BICM channel by m parallel binary-input continuous-output channels, the 
BICM capacity for a given constellation Vt is defined as 



m—1 



Iq 1 (SNR) = E Ic k {Ck\ Y) (32) 

PY\C k ,H(Y 



E E °k,H, 



k=0 
m—1 

log 2 

k=0 
m—1 

E E P cM®H, Y \C k =u 
k=0 «G{0,1} 

m—1 



Py\h(Y) _ 
log 



PY\H,C k =u(Y) 

Py\h{Y) 



(33) 
(34) 



n III -L n 

= p H (h)J2 E E P x{ x i) I PY\X= Xl ,H=h{y)- 

J^ N k=Oue{o,i}iex k , v JrN 

P^ M T. 3 ex k , ll Px{x, j )p Y \ x=Xj , H=h {y) 

22 xe x Px{x)PY\x= x ,H=h{y) 
where (1351) follows from (|34|) . (I3TI) . and the fact that the value of does not affect the conditional 

channel transition probability, i.e., p Y \x= x ,H=h,c k =u(y) = PY\x= x ,H=h(y)- The BICM capacity 
in (|35l) is a general expression that depends on all the constellation parameters Q. This can be 
numerically implemented using Gauss-Hermite quadratures, or alternatively, by using a one- 
dimensional integration based on the pdf of the L-values developed in 11351 . Il37l - [l39l . 

The AMIs Ic k {Ck]Y) in (T32l) are, in contrast to the ones in (|29l) . not conditioned on the 
previous bit values. Because of this, and unlike the CM capacity, the binary labeling strongly 
affects the BICM capacity l^(SNR) in (|32l . Note that the BICM capacity is equivalent to 
the capacity achieved by MLC with (suboptimal) parallel decoding of the individual bit levels, 
because in BICM, the bits are treated as independent (5TJ. The differences are that BICM uses 
only one encoder, and that in BICM the equivalent channels are not used in parallel, but time 
multiplexed. Again, following the standard terminologyLl used in the literature (cf. lfT3l . ([2T|. 
11231 . 11481 . ||49l ), we use the name "BICM capacity" even though no optimization over the input 
distribution is performed. 

If all the bits at the input of the modulator are equally likely, i.e., Pc k { u ) = 1/2 for k = 
0, . . . , m — 1 and u G {0, 1}, we obtain from d30l) Px( x ) = 1/M. Under these constraints, and 

6 It is also called parallel decoding capacity in 1261 . or receiver constrained capacity in 1461 . 
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assuming an AWGN channel (H = 1), the BICM capacity in (1351) is given by 

1^ (SNR) = — > > > / p m=a! .(?/)log 2 ^= ■ r^ dy > (36) 

where the constellation is f2 = [X, L, Um]- This expression coincides with the "standard" BICM 
capacity formula (cf. [EH Sec. 3.2.1], 031 eq. (15)], [01 eq. (11)]). 

One relevant question here is what is the optimum labeling from a capacity maximization 
point of view. Once this question is answered, approaching the fundamental limit will depend 
only on a good design of the channel encoder/decoder. Caire et al. conjectured the optimality of 
the BRGC, which, as the next example shows, is not correct at all SNR. This was first disproved 
in |]24| for PAM input alphabets based on an exhaustive search of binary labelings up to M = 8. 

Example 3 (CM and BICM Capacities for the AWGN channel, 8-PAM, and U 8 ): In Fig. [2} we 
show the BICM capacity in ([36]) and the CM capacity in ([25]) for 8-PAM, P = U 8 , and the four 
binary labelings in Example [TJ Fig. [2] (a) illustrates that the difference between the CM capacity 
and the BICM capacity is small if the binary labeling is properly selected. The best of the four 
binary labelings is the NBC for low SNR (#<, < 0.43 bit/symbol), the FBC for medium SNR 
(0.43 < R c < 1.09 bit/symbol), and the BRGC for high SNR (R c > 1.09 bit/symbol). Hence, 
the BRGC is suboptimal in at least 36% of the R c range. The gap between the CM capacity 
and the BICM capacity for the BSGC is quite large at low to moderate SNR. The low-SNR 
behavior is better elucidated in Fig. [2] (b), where the same capacity curves are plotted versus 
E h /N instead of SNR. Interestingly, the CM capacity and the BICM capacity using the NBC 
achieve the SL at asymptotically low rates; Gaussian inputs are not necessary, cf. [58, Sec. I]. 

Formally, E-^/Nq is bounded from below by f(R c ), where 

f(R C ) = i" r l™ ■ (37) 



®h\H 2 ]R c - 

This function always exists, because the capacity 7 ] C(SNR) is a strictly increasing^ function of 
SNR and thus invertible, while in contrast f(R c ) is in general not monotone. This is the reason 

7 From now on we will refer to "capacity" using the notation C(SNR) in a broad sense. C(SNR) can be the AWGN capacity 
C AW (SNR) in l[22j, the CM capacity l£ M (SNR) in (25}, or the BICM capacity lB : (SNR) in $32%. 

8 This can be proved using the relation between the AMI and the minimum mean square error (MMSE) presented in 1591 , 
i.e., that the derivative of the AMI with respect to SNR is proportional to the MMSE for any SNR. Since the MMSE is a strictly 
decreasing function of SNR, the AMI is a strictly increasing function of SNR. 
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Figure 2. CM capacity and BICM capacity for 8-PAM with Ug using the four labelings defined in Sec. III-Bl plotted vs. SNR 
(a) and Eh/No (b), and their corresponding functions g(R c ) (c). The shadowed regions represent the achievable rates using the 
BSGC. The black squares represent the minimum Eb/ 'No for the BSGC. The black circles represents the Eb/ No needed for a 
rate 7? = 1/15 turbo code to reach BER = 1CT 6 (cf. Sec. PVl-Al l. 
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why a given E h /N for some labelings maps to more than one capacity value, as shown in 
ll2TI . The phenomenon can be understood by considering the function l BI (SNR) in a linear SNR 
scale, instead of logarithmic as in Fig. |2] (a). If plotted, the function would pass through the 
origin for all labelings. Furthermore, any straight line through the origin represents a constant 
~E H [H 2 ]E h /N by ©, where the slope is determined by the value of ~E H [H 2 ]E h /N . Such a line 
cannot intersect l BI (SNR) more than once for SNR > 0, if l BI (SNR) is concave. This is the case 
for the BRGC, NBC, and FBC, and therefore the function R c = f~ 1 (¥ JH [H 2 ]E h /N ) exists, as 
illustrated for in Fig. [2] (b). However, for some labelings such as the BSGC (and many others 
shown in ll49l Fig. 3.5]), l BI (SNR) is not concave and f(R c ) is not invertible. This phenomenon 
has also been observed for linear precoding for BICM with iterative demapping and decoding 
ll60l Fig. 3], punctured turbo codes [61, Fig. 3], and incoherent M-ary PSK |[62l Figs. 2 and 5] 
and frequency-shift keying channels ll63l Figs. 1 and 6]. 

Since analytical expressions for the inverse function of the capacity are usually not available, 
expressions for f(R c ) are rare in the literature. One well-known exception is the capacity of the 
Gaussian channel given by ([22]) . for which 

/ AW W = ^(2 2 ^-1), (38) 

which results in the SL 

\im + f AW (R c ) = log e (2) = -1.59 dB. (39) 

Analogously, we will use the notation /^ M (i? c ) and fQ l (R c ) when the capacity considered is 
the CM and the BICM capacity, respectively!^ 

The results in Fig. |2] (a)-(b) suggest a more general question: What are the optimal constel- 
lations for BICM at a given SNR? To formalize this question, and in analogy to the maximum 
CM capacity in (|28l ), we define the maximum BICM capacity as 

C BI (SNR) 4 max l BI (SNR) , (40) 

where the optimization is in this case over the three parameters defining fl In analogy to the 
maximum CM capacity, the maximum BICM capacity represents an upper bound on the number 

9 The same notation convention will be used for other functions that will be introduced later in the paper. 
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of bits per symbol that can be reliably transmitted using a fully optimized BICM system, i.e., a 
system where for each SNR, the constellation is selected to maximize the BICM capacity. 

We conclude this subsection by expressing the BICM capacity as a difference of AMIs and 
conditional AMIs , which will facilitate the analysis in Sec. [IV] The following result is a somehow 
straightforward generalization of ET1 Proposition 1], j|47l eq. (65)] to iV-dimensional input 
alphabets, nonuniform input distributions, and fading channels. 

Theorem 2: The BICM capacity can be expressed as 

m— 1 

ff(SNR) = X; Pc k (u)[lx(X;Y)-I x]Ck =u(X;Y)]. (41) 



fc=0 mG{0,1} 

Proof: For any function e(X,Y, H), 



~^H,Y\C k =u 



, PY\HC k =u(X) 

log 2 — — 

Vy\h(X) 



^X,Y,H\C k =u 



e(X,Y,H) e(X,Y,H) 

lo S2 7^ l0 S2 



PY\H(Y) PY\H,C k =u(Y) 



Using this relation in ([34]), letting e(X, Y, H) = Py\x,h(Y) = p Y \x,H,c k =u(Y), observing 
that the first term is independent of u, and utilizing <[T9l ) and ([24]) yields the theorem. □ 

E. Minimum E^/Nq for Reliable Transmission 

In this section, we determine the minimum E^/Nq that permits reliable transmission, for a 
given input alphabet and labeling. As observed in Fig. [2] (b), this minimum does not necessarily 
occur at rate R c = 0. 

Theorem 3 (Minimum E^/Nq): The minimum E^/Nq is given by f(R c ), where R c = or R c 

is one of the solutions of g(R c ) = 0, where 

g(Rc) A #W = 1 dC~\R c ) _ C-\R C ) ^ (42) 
dR c R c dR c R^ 

Proof: Any smooth function has a minimum given by the solution of its first derivative 

equal to zero or at the extremes of the considered interval. Since in general < R c < oo, two 

extreme cases should be considered. However, lim^^oo f AW (R c ) = \im Rc ^. m - fn(R c ) = oo, 

and therefore, the only extreme point of interest is R c = 0. □ 

Since f(R c ) is in general not known analytically, the function g(R c ) must be numerically 

evaluated using C(SNR). An exception to this is the capacity of the AWGN channel, where 

g AW (R c ) can be calculated analytically. Moreover, it can be proved that in this case, a minimum 

E h /N for nonzero rates does not exist. 
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Corollary 4 (Minimum E h /N for the AWGN channel): The minimum E h /N for the AWGN 
channel is unique, and it is obtained for zero-rate transmissions. 
Proof: The derivative of f AW (R c ) in (1381 ) is given by 

aw,„ , /V + (2ft log, 2-^)2"^" 4 <g,(ft) 

9 W = 2*2 = 5Sft? <43) 

To prove that a zero for a nonzero rate does not exit, we need to prove that ^*(fl c ) > for 
R c > 0, since g^(R c ) > for R c > 0. This follows because \im Rc ^, + g^L(Rc) = and the 
first derivative of ^^(-R c ) is strictly positive: 

d9num( R c) _ A^ c (l g c 2) 2 2 2Rc/jV > 0. 

ari c iv 

□ 

In Fig. [2] (c), we present the function g(-R c ) in (l42l) for the same constellations presented 
in Fig. [2] (a)-(b). If g(R c ) = has at least one solution for R c > 0, the capacity curve will 
have a local minimum (shown with a filled square in Fig. [2] (b)-(c) for the BSGC). Note also 
that the BSGC has an interesting property, namely, lim Rc ^ + 9q 1 (Rc) — —oo, and consequently, 
lim fic ^ + fn l {Rc) = +oo. In this sense, the BSGC is an extremely bad labeling for M-PAM 
input alphabets and asymptotically low rates. 



F. Probabilistic Shaping 

The maximum BICM capacity in (l40l) is an optimization problem for which analytical solutions 
are unknown. In this subsection, we study the solution of (l40l) when the input alphabet and the 
binary labeling are kept constant, i.e., we study the so-called probabilistic shaping. Formally, we 
want to solve P*(SNR) = argmax P l^ 1 (SNR), where Q = [X, L, P], for a given input alphabet X 
and labeling L. Since this optimization problem turns out to have multiple local minima and no 
analytical methods are known for solving it, we perform a grid search with steps of 0.01 based 
on Gauss-Hermite quadratures. The optimization is performed over the three variables defining 
the input distribution: Pc L (0), Pc 2 (0), and Pc 3 (0). For each SNR value, the input distribution 
that maximizes the BICM capacity is selected. 

In Fig. |3l we show the BICM capacity for an 8-PAM input alphabet labeled by the BRGC and 
the NBC, when the optimized input distributions are used. We use the notation f2* = [X, L, P*]. 
The results in this figure show how, by properly selecting the input distribution, the BICM 
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capacity can be increased. The gap between the BICM capacity and the AWGN capacity is 
almost completely eliminated for R c < 2 bit/symbol (in contrast to a gap of approximately 
1 dB in Fig. |2](b)). Similar results have been presented recently in ||64ll for 4-PAM. Interestingly, 
Fig. [3] shows that if the input distribution is optimized, the NBC is not the optimal binary labeling 
for low SNR anymore, but the BRGC with an optimized input distribution achieves the SL. This 
is also the case for the FBC, but we do not show those results not to overcrowd the figure. 

IV. BICM for Asymptotically Low Rates 

In this subsection, we are interested in finding an asymptotic expansion for the CM and the 
BICM capacities when SNR -> 0. 

A. Relation between AWGN and BICM capacity 

We start by proving that the BICM capacity can be optimal in the sense of being equal to the 
AWGN capacity only for zero rate. This very simple result motivates the developments in order 
to characterize the behavior of BICM for asymptotically low SNR. 

Theorem 5: The AWGN capacity, the CM capacity, and the BICM capacity are related through 
the following two inequalities. 
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i) l£ M (SNR) < C AW (SNR) with equality if and only if SNR = 0, and 

ii) I* 1 (SNR) < I™ (SNR). 

Proof: We start by proving that l AW (SNR) < C AW (SNR) for SNR > 0, where l£ w (SNR) 
is the CM capacity of the AWGN channel. From © and I X (X;Y) = h(Y) - h{Z), we 
express the CM capacity for a given Vt in terms of differential entropies as l^ w (SNR) = 
h(Y) — iV/ 2 log 2 (27r7Voe). Since the differential entropy h(Y) = — p Y (y) log 2 py iy) dy 



is maximized if and only if Y is Gaussian distributed Il44l Theorem 8.6.5], the use of any 
constellation f2 (discrete input alphabet) will give a smaller differential entropy h(Y) than for 
a Gaussian Y, which proves that l AW (SNR) < C AW (SNR) for SNR > 0. 

We now prove that lg M (SNR) < l£ w (SNR) for any fading channel and SNR > 0. To do this, 
we note that the CM capacity for fading channels is equal to the CM capacity for the AWGN 
channel averaged over the distribution of the instantaneous SNR. Furthermore, Iq W (SNR) is a 
strictly concave function of SNR for SNR > 0, because the second derivative of the AMI as a 
function of the SNR (the first derivative of the MMSE, see footnote 8) is strictly negative for 
SNR > ll65l Proposition 5] ll66l Proposition 7]. Therefore, Jensen's inequality holds, which 



yields l£ M (SNR) = E H [l£ w (H 2 E S /N )} < l AW (E H [H 2 }E S /N ) = l AW (SNR) for SNR > 0. 



This and the fact that l^ M (0) = C AW (0) = proves item i). The proof of item ii) was presented 



Corollary 6: The BICM capacity and the maximum BICM capacity can be equal to the 
AWGN capacity only for zero rates, i.e., I^ 1 (SNR) = C BI (SNR) = C AW (SNR) only if SNR = 0. 

Proof: From Theorem [51 we know that for any SNR > 0, the inequality l^(SNR) < 
l£ M (SNR) < C AW (SNR) holds. Therefore, for any SNR > 0, l^ 1 (SNR) < C AW (SNR). The 
proof for the BICM capacity is completed noting that l BI (0) = C AW (0) = 0. The proof for the 
maximum BICM capacity follows from the fact that Theorem \5\ holds also when an optimization 
over Cl is applied. □ 

Corollary [6] simply states that the only rates for which the AWGN will be equal to the BICM 
capacity and the maximum BICM capacity is R c = (or equivalently SNR = 0). In the following 
subsections, we analyze the asymptotic behavior of the BICM capacity when SNR = 0. 




in El Sec. III]. 



□ 
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B. A Linear Approximation of the Capacity and the SL 

Any capacity function C(SNR) can be approximated using a Taylor expansion around SNR = 
as C(SNR) = ctSNR + 0(SNR 2 ). By inversion of power series (671 Sec. 1.3.4.5], we find 

Q-\R c ) = -R c + 0(R 2 c ), 
a 

and using (l37l) . it is possible to obtain a linear approximation of the function f(R c ) 

f(R c ) = - + 0(R c ). (44) 

a 



For asymptotically low rates, (1441) results in 

lim f(Rc) = -, (45) 
and since from (1391) l/a > log e (2), we obtain 

"<log 2 e. (46) 



It is clear from d46 



the SL -1.59 dl 



J 10 



that a capacity function C(SNR) that has a coefficient a = log 2 e achieves 
Moreover, based on the results for the BSGC in Fig. |2] (b), the coefficient 



a can be as low as zero. 

C. First-Order Asymptotics of the BICM Capacity 

Theorem 7 (Linear approximation of the AMI): When the channel is perfectly known at the 
receiver, and for any input distribution Px(x), the AMI between X and Y in © can be 
expressed as 

I X (X; Y) = aSNR + 0(SNR 2 ) 

when SNR — > 0, where 

a = log 2 e 1- 11 X l J " ). (47) 



Proof: The proof is given in Appendix |Al □ 
Theorem [7] shows how to calculate the first-order asymptotics of an AMI with arbitrary input 
distribution. The following corollary follows directly from the definition of the CM capacity in 
(T25T) . where the input distribution is given by (l30l) . 

10 Or equivalently, if we measure the AMI in nats, a = log c e = 1. 
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Corollary 8 (Coefficient a^ M ): The CM capacity can be expressed as 

I™ (SNR) = ag M SNR + 0(SNR 2 ) 



ri _ lo §2 e 



BI 



when SNR ->> 0, where a^ M is given by (|47|). 

The next theorem gives the first-order asymptotics for the BICM capacity. 

Theorem 9 (Coefficient a® 1 ): The coefficient a for the BICM capacity l|^(SNR) is given by 

m— 1 

E E Pc k (n)\\E xlCk=u [X]\\ 2 - m\\E x [X]\\- 

k=0 u£{0,l} 

Proof: Reordering the result of Theorem |2l we have that 

m— 1 ^ 

lH I (SNR) = E |/x(X;Y)- E Pc fc H/ X | Cfc=M (X;r) 
fe=o ^ ue{o,i} 

Since Ix(X; Y) and I x \c k =u{X] Y) are AMIs, we can apply (1471) to each of them, which gives 
^E{^-|I E *™ 2 - E ^H(E x|Cfc=u [||X|| 2 ]-||Ex|c fc = u [X]|| 2 )}. 

S k=0 ^ «G{0,1} 

We recognize X!«e{o 1} -Pc* fe ( M )Ex|c fe =«[||^ || 2 ] a s the average symbol energy E s , which com- 
pletes the proof. □ 

The first-order coefficients of the expansion of the CM and BICM capacities in Corollary [8] and 
Theorem [9] do not depend on the fading. This simply states that, under the constraints imposed 
on H, the fading has no effect on the first-order behavior of the BICM capacity. Consequently, 
the analysis of the optimal constellations for fading channels at low SNR can be reduced, without 
loss of generality, to the AWGN case. 

Corollary [8] and Theorem [9] generalize the results in J2TJ, [|22l| by considering constellations 
with nonuniform input distributions and arbitrary dimensions, mean, and variance. This gener- 
alization will allow us to analyze optimal constellations fl in the next section. 

In general, we know from (|46|) that a® 1 < log 2 e, which can be interpreted as the penalty 
of a certain BICM system over an optimal CM system (without interleaving). In the following 
section we analyze ct^ 1 for PAM and PSK input alphabets with different binary labelings and 
P = Ujvf and we also show how to obtain a^ 1 = log 2 e for general constellations. 
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V. First-Order Optimal Constellations for BICM 



Shannon stated in 1959, "There is a curious and provocative duality between the properties of 
a source with a distortion measure and those of a channel" ll68l . Many instances of this duality 
have been observed during the last 50 years of communications research. A good summary of 
this is presented in [45, Sec. V]. The coefficient a is mathematically similar to the so-called 
linearity index HDl . which was used to indicate the approximative performance of labelings in 
a source coding application at high SNR. The usage of the HT in this section was inspired by 
the analysis in ||40l . 



A. FOO Constellations 



n 



In view of the SL (l46l) . we define & first-order optimal (FOO) constellation for BICM_J as a 



constellation Vt that results in a coefficient a® 1 = log 2 e. 

„bi 



Theorem 10 (Coefficient for arbitrary constellations): For any constellation Q 



a 



BI 



2E 9 



m—l 



M-l 



k=0 I i=0 V P C k ( C i,k) 



M-l 



E 



XiP X {Xi) 



2 E X \X] 



(49) 



where are the elements of the modified labeling matrix in ©• 

Proof: The proof is given in Appendix [Bj □ 

Theorem [TOl is a very general theorem valid for any constellation f2. From this theorem, it is 
clear that the problem of designing FOO constellations for BICM has three degrees of freedom: 
the input alphabet X, the binary labeling L, and the input distribution P. 

From now on, we restrict our attention to uniform input distributions P. This restriction can 
be justified from the fact that due to the digital implementation of the transceivers, changing 
the input alphabet or the binary labeling can be implemented without complexity increase. On 
the other hand, implementation of probabilistic shaping requires a modification of the channel 
encoder and/or the interleaver. If P = U M , then P Ck (u) = 1/2 for k = 0, 1, . . . ,m — 1 and 
u E {0,1}, and (l49l) simplifies into 



BI 



, m—l 

log 2 e 



fc=0 



^ M-l 

i=0 



(50) 



n A similar first-order optimality criterion for the CM capacity can be defined. In this case, based on {47}, any constellation 
based on a zero-mean input alphabet is an FOO constellation for the CM capacity, regardless of the input distribution Px(x). 
Conversely, no FOO constellation can have nonzero mean. 
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Keeping X fixed and changing the labeling L is equivalent to fixing L and reordering the 
rows of X. Therefore, a joint optimization of f2 = [X, L, Um] over X and L can without loss 
of generality be reduced to an optimization over X only, for an arbitrary L. In the following 
analysis, we will hence sometimes fix the labeling to be the NBC, without loss of generality. 

The expression for a^ 1 in (l50l) can be simplified further using the HT, as elaborated in the 
next theorem. 

Theorem 11 (The HT and a® 1 ): The coefficient a® 1 for a constellation f2 = [X, N m , Um] is 
given by 

, m— 1 

^' = ^£lNMI ! . 

s k=0 

where x 2 k are elements of the HT of X defined by (PT4l) . 
Proof: Using Lemma Q] and (fl"5l) in <l50l ), we obtain 

AI-l 

M 



m—l 

m lo S2e 



£ 



s k=0 



77 Yl h >* x " 



i=0 



m-1 



log 2 e x ^ 2 
> \\Xok \\ . 

E II 2 II 

s fc=0 



□ 



^<^Ell^l| 2 = ^ElNI| 2 = log 2 e (51) 



It follows from Theorem [TT] and (TT6l) that 

, M-1 

log 2 e ^ 2 _ log 2 e 
^ l|£Cjl1 ~ ME S 

j=0 s i=0 

for any constellation, which is in perfect agreement with (l46l) . We now proceed to determine 
the class of input alphabets and labelings for which the bound (I5TT) is tight. 

Theorem 12 (Linear projection of a hypercube): A constellation Q = [X, L, Um] is FOO if 
and only if there exists anmxJV matrix V = [v^, . . . , v^^} 1 - such that 

X = Q(L)V. (52) 



Proof: Consider first the NBC. Equality holds in (1511) if and only if Xj = for all j 
0, . . . , M - 1 except j = 1, 2, 4, . . . , 2 m ~ 1 . For such input alphabets, (Q~5]) yields 



m— 1 

JEj = ^ ^ Hi 2 k X2 k - 
k=0 

Letting v k = i 2 fc for = 0, . . . , m — 1 and using (fl~3T) . we obtain 

m— 1 

5^gi,fc^fc, i = 0,..., M-1. (53) 



m—l 
k=0 
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Oil 001 




110 100 010 — Oil 

(a) OTTO constellation (b) OTOTO constellation 



Figure 4. The two FOO constellations defined in Example [4] (m = 3 and TV = 2). Graphically, the OTTO constellation in (a) 
gives the impression of a projected cube. The OTOTO constellation in (b) gives the impression of a 6-PSK input alphabet with 
two extra points located at the origin. 



Letting V = [vq, . . . , completes the proof for L = N m . That the theorem also holds for 

an arbitrary labeling follows by synchronously reordering the rows of X and L, as explained 
before Theorem \TT\ □ 

Theorem [121 has an appealing geometrical interpretation. Writing the set of constellation points 
as in (1521 . each row of Q can be interpreted as a vertex of an m-dimensional hypercube, and 
VasanmxAf projection matrix. Hence, a constellation for BICM is FOO if and only if its 
constellation is a linear projection of a zero-mean hypercube. This interpretation, as well as all 
theorems presented so far, holds for an arbitrary dimension N. In the rest of this section, we 
will exemplify the results for N = 1 and 2, because such input alphabets are easily visualized 
(Figs. and often used in practice (PAM, QAM, and PSK). 

Example 4 (OTTO and OTOTO constellations): To exemplify the concept of Theorem [T2l we 
present two constellations that are FOO. The projection matrices for the "one-three-three-one" 
(OTTO) and the "one-two-one-two-one" (OTOTO) constellations are defined as 



VqTOTO 



V 



OTTO 



-1 -1 
+ 1 
-1 +1 



cos(7r/3) sin(7r/3) 
cos(7r/3) — sin(7r/3) 



Both constellations are shown in Fig. 0] The figure illustrates that the minimum Euclidean dis- 
tance, which is an important figure-of-merit at high SNR, plays no role at all when constellations 
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000 001 010 Oil 100 101 110 111 



Figure 5. Hierarchical 8-PAM constellation. The constellation is FOO and V = [—do, —di, ■ ■ ■ , — d m _i] T . 



are optimized for low SNR. 

A particular case of Theorem [T2l are the nonequally spaced (NES) M-PAM input alphabets, 
as specified in the following corollary. 

Corollary 13: If a NES M-PAM input alphabet X consists of the points ±v ±Vi±- ■ -±v m -i, 
there exists a binary labeling L such that the constellation [X, L, Um] is FOO. 

Example 5 (Hierarchical constellations): The so-called "hierarchical constellations" ll69l - ir7Ti 
are defined by the one-dimensional input alphabet [69, eq. (3)] 

m— 1 

Xi = 5](26 fc (i)-l)d fcj (54) 

fc=0 

where bk(i) is the base-2 representation of the integer i with i = 0, . . . , M — 1, and where 
dk > for k = 0, . . . , m — 1 are the distances defining the input alphabet. The additional 
condition < x i+i for % = 0, . . . , M — 2 is usually imposed so that overlapping points in 
the input alphabet are avoided. This condition also keeps the labeling of the input alphabet 
unchanged. 

In Fig. [51 we show a hierarchical 8-PAM input alphabet. In this figure, the M constellation 
points are shown with black circles, while the white squares/triangles represent 2- and 4- 
PAM input alphabets from which the 8-PAM input alphabet can be recursively (hierarchically) 
constructed. 

The binary labeling used in hierarchical constellations is usually assumed to be the BRGC. 
In this case, we find that when X is given by (l54l) . the system in (|52|) has no solutions for 
V, and therefore, the constellation is not FOO. However, if the NBC is used instead (as in 
Fig. [5]), all hierarchical constellations are FOO, because X = Q(N m ) V gives a projection matrix 

V = [-d , ~di, • • • , -gL_i] t . 
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B. Labelings for PAM, QAM, and PSK 

While we have so far kept the labeling fixed and searched for good input alphabets, we now 
take the opposite approach and search for good labelings for a given input alphabet. In this 
section we analyze the practically relevant input alphabets PAM, QAM, and PSK defined in 
Sec. III-BL Throughout this section, we assume P = Um- 

Example 6 (NBC for M-PAM): Let V = [vq, v u . . . , w m _i] T = [-1, -2, -4, . . . , -2 m - x ] T 
and let L = N m . With given by (fl"3l) . we obtain from (|52|) the constellation Xpam — 
[-M + 1, -M + 3, . . . , M - 1] T , which shows that the constellation [X PAM , N m , U M ] is FOO. 
In view of Theorem CCD the optimality of M -PAM input alphabets comes from the fact that the 
HT of Xpam has its only nonzero elements in the m positions 1, 2, 4, ... , 2 m ~ 1 . 

It follows from Example [6] that the constellation [Xpam, N m , Um] is FOO, which has also 
been shown in [|23l . The following theorem states that the NBC is the unique labeling with this 
property, apart from trivial bit operations that do not alter the characteristics of the labeling. 

Theorem 14: The constellation [Xpam, L, Um] is FOO if and only if L = N m , or any other 
binary labeling that can be derived from the NBC by inverting the bits in certain positions or 
by permuting the sequence of bits in all codewords. 

Proof: The proof is given in Appendix O □ 

In order to extend this result to rectangular QAM constellations, we first state a theorem about 
product constellations in general. 

Theorem 15: A two-dimensional constellation [X,L,Um], where X = X' £g)X" is the ordered 
direct product of two one-dimensional input alphabets X' and X" and all symbols Xi are distinct, 
is FOO if and only if both the following items hold. 

. There exist labelings L' and L" such that [X',V,Um>] and [X", L", Um"] are both FOO 

(where M' and M" are the sizes of X' and X", resp.). 
• L = nc(L' ® L"), where lie is an arbitrary column permutation. 

Proof: The proof is given in Appendix [Ql □ 
As a special case, the theorem applies to rectangular QAM constellations since they are defined 
as the ordered direct product of two PAM input alphabets. In view of Theorem Q31 and since 
N m > ® N m » = N m / +m », the following corollary gives necessary and sufficient conditions for a 
rectangular (M' x M")-QAM constellation to be FOO. 
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Corollary 16: A constellation [Xq AM ,L, U^], where Xq A m is an (M' x M")-QAM input 
alphabet and M = M'M" = 2 m , is F00 if and only if L = N m , or any other binary labeling 
that can be derived from N m by inverting the bits in certain positions or by permuting the 
sequence of bits in all codewords. 

Can a constellation based on an Af-PSK input alphabet be FOO with a suitably chosen 
labeling? What about constant-energy constellations in higher dimensions? A complete answer to 
these questions is given by the following theorem. An intuitive interpretation is that a constellation 
based on a constant-energy input alphabet is FOO if and only if it forms the vertices of an 
orthogonal parallelotope, or "hyperrectangle." 

Theorem 17: A constellation [X, L, Um], where \\xi\\ 2 is constant for alH = 0, . . . , M — 1, is 
FOO if and only if X can be written in the form (|52|) with orthogonal vectors v , . . . , f m _i. 
Proof: The proof is given in Appendix El □ 

The case of PSK input alphabets follows straightforwardly as a special case of Theorem [FT] 
Indeed, the fact that a set of m orthogonal vectors cannot exist in fewer than m dimensions 
leads to the following conceptually simple corollaries. 

Corollary 18: FOO constellations based on constant-energy input alphabets in N dimensions 
cannot have more than 2 N points. 

Corollary 19: No FOO constellations based on A/-PSK input alphabets exist for M > 4. 

Observe that the criterion in Theorem [FT] is that v , . . . , v m -\ should be orthogonal, not 
necessarily orthonormal. Thus, FOO constellations based on constant-energy input alphabets are 
not necessarily hypercubes. In particular, a 4-PSK input alphabet does not have to be equally 
spaced to give an FOO constellation. Indeed, any rotationally symmetric but nonequally spaced 
4-PSK input alphabet (i.e., a rectangular one) gives an FOO constellation. 

C. M-PAM and M-PSK Input Alphabets 

In this subsection, we particularize the results in Sec. IIV-BI to practically relevant BICM 
schemes, i.e., Af-PAM and Af-PSK input alphabets with uniform input distributions using the 
four binary labeling s defined in Sec. III-BI 

Theorem 20 (Coefficient a® 1 for f2 = [Xpam, L m , Ua/]J: For M-PAM input alphabets using 
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Ujvf, the coefficient ct^ 1 for the binary labelings defined in Sec III-BI is given by 

3M 2 



4(M 2 -l) l0g2e ' ^ 

log 2 e, if 

0, if L ri 

Proof: The proof is given in Appendix [0 



or L.„ 



F 



(55) 



□ 



Theorem 21 ( Coefficient a® 1 for H = [X PSK , L m , V M ]): For M-PSK input alphabets using U M , 
the coefficient a^ 1 for the binary labelings defined in Sec III-BI is given by 



CVf 



BI 



8 log 


5 2 e 




M 2 sin 2 
4 log 


(tt/M)' 
5 2 e 




M 2 sin 2 
4 log 


(tt/M) ' 
? 2 e 


"l+(l-secfff 

m 

l + ^tan 2 (7r/2 fe 
fc= 2 


M 2 sin 2 
4 log 


(tt/M) 
? 2 e 


M 2 sin 2 

V 


(tt/M) 



if L,; 
if L,; 
if L,; 

if m 



G r 



(56) 



F 



where sec x = 1/ cos x is the secant function. 

Proof: The proof is given in Appendix [Gl □ 

In Fig. [6l we present the pmf of the coefficient a® 1 obtained via an exhaustive enumeration 
of the 8! = 40320 different binary labelings (without discarding trivial operations) for 8-PAM 
and 8-PSK with Us. For 8-PAM, Fig. [6] (a) shows that many binary labelings are better than the 
BRGC at low SNR, the best one being the NBC as found in ll23l . On the other extreme we find 
the BSGC, which gives a coefficient equal to zero, reflecting the inferior performance in Fig. |2] 
(b). Based on (|45T) . we obtain that the E^/Nq for reliable transmission at asymptotically low 
rates in this case is oo, and it is independent of M. We find that among the 8! possible binary 
labelings, there exist 72 classes of binary labelings that have a different a® 1 , and therefore, a 
different first-order asymptotic behavior. We also note that the BICM capacity for the BRGC 
and the FBC in Fig. [2] (b) are different for SNR > 0. However, their coefficient a® 1 in (1551) is 
the same, and thus, the curves for these labelings in Fig. [2](b) merge at low rates. 

Fig. [6] (b) shows that for 8-PSK, there exist only 26 classes of binary labelings with different 
coefficients a^ 1 . In particular, the NBC and the BSGC result in a moderate coefficient, and the 
BRGC in a quite high coefficient. We found that the FBC is the asymptotically optimal binary 
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Figure 6. The pmf of a™ for 8-PAM (a) and 8-PSK (b) with Ug. The four labelings defined in Sec. uTBI are shown with 
white markers. 



labeling for 8-PSK, unique up to trivial operations, and we conjecture it to be optimal for any 
M-PSK input alphabet and m > 2. Interestingly, there are no binary labelings for 8-PSK that 
give a coefficient zero or one, and the number of distinct pmf values is only ten (25 for 8-PAM). 

From (145T) we know that off determines the behavior of the function /^(Rc) for asymptotically 
low rates. Following the idea introduced in [|2T|. we analyze how the values of aj] 1 for PAM 
and PSK input alphabets behave when M — > oo. A summary of the values of lim a?, 1 and 

M->oo 

lim /o^-Rc) for M-PAM and Af-PSK input alphabets using U M are presented in Table 1 for 
the four labelings previously analyzeco. For most of the constellations, there is a bounded loss 
with respect to the SL when M ->■ oo. For the BRGC, this difference is 1.25 dB for M -PAM 
and 0.91 dB for Af-PSK. On the other hand, for the NBC and M-PAM, the difference is zero 
for any M. Note that all the coefficients a® 1 in (1551) and in (l56l) are nonincreasing functions of 
M. 

1 2 BI 1 / Al ^ 1 

'The limits lim a n for A/-PSK are obtained based on liniM-^ -r-k — tjt = -7 (obtained by L'Hopital's rule). For the 
NBC, we obtain numerically that 2 tan 2 (Tv/2 k ) w 1.2240, which gives the coefficient 8.89 in Table U 
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Table I 

First-order asymptotics of M-PAM and M-PSK input alphabets using U m for different binary labelings. 







PAM 




PSK 




l- BI 

lim a n 

M — > oo 


lim fR\Rc) 

i? c ->0+ 


lim otn 

M— >oo 


lim fS\R, 

R c ->0+ 




I log 2 e 


-0.34 dB 


log 2 e 


-0.68 dB 


N m 


log 2 e 


-1.59 dB 


£ log 2 e 


2.33 dB 







oo 


£ log 2 e 


2.33 dB 




f log 2 e 


-0.34 dB 


^log 2 e 


-1.14 dB 



VI. Numerical Examples 

A. Turbo-coded System Simulation 

In order to validate the analysis presented in the previous sections, we are interested in 
corroborating if the use of the NBC instead of the BRGC for PAM input alphabets actually 
translates into a real gain when capacity-approaching codes are used. To this end, we simulate a 
BICM scheme which combines a very low rate capacity-approaching code with M-PAM input 
alphabets. We use Divsalar's rate- 1/1 5 turbo code, formed by a parallel concatenation of two 
identical 16-state rate- 1/8 recursive systematic convolutional (RSC) convolutional codes defined 
by their polynomial generators (1, 21/23, 25/23, 27/23, 31/23, 33/23, 35/23, 37/23) 8 02. The 
two RSCs are separated by a randomly generated interleaver of length N = 16384, and 64 tail bits 
are added to terminate the trellis, giving an effective code rate of R = 16384/ (15 • 16384 + 64). 
We combine this turbo code (via a randomly generated interleaver) with 4-PAM and 8-PAM 
using NBC or BRGC, yielding R c fa 0.13 bit/symbol and R c ~ 0.2 bit/symbol respectively. 
The constellation symbols are equally likely, the decoder is based on the Log-MAP algorithm, 
and it performs 12 turbo iterations. In Fig. [71 the bit error rate (BER) performance of such a 
system is presented. 

We study the E h /N needed for the four different constellations to reach a BER target BER = 
10~ 6 . For 4-PAM, the values for the BRGC and the NBC are, respectively, E h /N = 0.99 dB and 
E h /N = 0.29 dB, i.e., the NBC offers a gain of 0.4 dB compared to the BRGC. For 8-PAM, 
the obtained values are E h /N = 1.05 dB and E h /N = 0.45 dB, which again demonstrate 
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Figure 7. BER for the rate-1/15 turbo code with 4-PAM and 8-PAM for the BRGC and the NBC (R c « 0.13 bit/symbol and 
R c ~ 0.2 bit/symbol respectively). The metrics' computation is based on (8), the interleaver size is N = 16384, the decoder 
is based on the Log-MAP algorithm, and it performs 12 turbo iterations. The filled circles represent the E^/No needed for the 
configuration to reach a BER = 10~ 6 , which are also shown for 8-PAM in Fig.|2](b). 



the suboptimality of the BRGC in the low SNR regime. Moreover, we also simulated an 8- 
PAM input alphabet labeled by the BSGC. We obtained in this case E h /N = 8.40 dB, i.e., a 
degradation of 7.95 dB is caused by a bad selection of the binary labeling. The values of E h /N 
obtained for these last three cases are shown in Fig. [2] (b). These results show that the turbo- 
coded system performs within 1 dB of capacity, and that the losses of 0.6 dB and 7.95 dB 
can be observed from the capacity curves as well. This indicates that the results obtained from 
Fig. |2] for different labelings can be used as an a priori estimate of the system performance 
when capacity-approaching codes are used. 

B. Capacity vs. E b /N 

In Fig.|g](a), we show the function f AW (R c ) and f£ M (R c ), defined in Sec. MM using 4-PAM 
and 8-PAM input alphabets. We also show /^(-Rc) for 4-PAM and 8-PAM input alphabets for 
different binary labelings and for hierarchical 4-PAM and 8-PAM constellations (Example [5]). The 
curves in Fig. [8] (a) intersect the horizontal axis at E h /N = 1 /an, where l/a AW = l/log 2 e = 
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Figure 8. (a) AWGN capacity, CM capacity, and BICM capacities for M -PAM with the BRGC, NBC and FBC. The BICM 
capacity for hierarchical 4-PAM with V = [—1, — 5] T and 8-PAM with V = [—1, —2, — 6] T is also shown. The white circles 
give the performance at R c = 0, where a^ 1 determines the BICM capacity. The BRGC and FBC are equivalent for M = 4. 
(b) SNR gap An(i? c ) in {57} for the same capacities and for 16-PAM. 



— 1.59 dB represents the SL. From this figure, we observe that for CM both constellations are 
FOO, while for BICM only four of them are FOO, the ones labeled by the NBC. 

In Fig. [9] (a), similar results for 8-PSK are shown. We also include the results for the OTTO 
and OTOTO constellations in Fig. 0] (Example B}. From this figure, we observe that for the CM 
capacity the 8-PSK input alphabet gives an FOO constellation, and for BICM, the OTTO and 
the OTOTO constellations are FOO. Moreover, for high SNR, the OTTO constellation results in 
a higher capacity than the constellations based on 8-PSK input alphabets. 

C. The SNR Gap 

Borrowing the idea from ll26ll . we define the SNR gap as the horizontal difference^ between 
the CM and BICM capacity and the capacity of the AWGN channel for a given R c , i.e., 

13 The gap is the same regardless of whether the horizontal axis represents E^/No or SNR. 



December 9, 2010 



DRAFT 



37 




Figure 9. AWGN capacity, CM capacity, and BICM capacities for 8-PSK with the BRGC, NBC, FBC and BSGC. The BICM 
capacity for the OTTO and OTOTO constellations are also shown. The white circles give the performance at R c = 0, where 
oq 1 determines the BICM capacity, (b) SNR gap An(R c ) for the same capacities. 

These expressions, which represent the additional energy needed for a given constellation to 
achieve the same R c as the optimal scheme (the AWGN capacity), are evaluated numerically 
in Figs. [8] (b) and [9] (b). In Table UH we present a summary of the SNR gap at asymptotically 
low rates for different constellations. This asymptotic SNR gap is given by log 2 e/«Q I , and is a 
scaled special case of the results presented in Sec. IV-CI 

VII. Conclusions 

In this paper, we introduced a general model for BICM which considers arbitrary input 
alphabets, input distributions, and binary labelings, and we analyzed different aspects of the 
BICM capacity. Probabilistic shaping for BICM was analyzed and the relation between the 
BICM capacity and E h /N was studied. Four binary labelings (BRGC, NBC, BSGC, and FBC) 
were analyzed in detail, and for 8-PAM with uniform input distribution, the results showed that 
the BICM capacity is maximized by the NBC and the FBC for 36% of the rates. 

First-order asymptotic of the BICM capacity for arbitrary constellations were presented, which 
allowed us to analyze the behavior of the BICM capacity for low rates. The E h /N required 
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Table II 

The SNR gap at asymptotically low rates for BICM and different constellations. 



Constellation \og 2 e/o^ [dB] 



4-PAM 


BRGC/FBC 


0.96 




NBC 





4-PAM Hierarchical 







BRGC 


0.69 


8-PSK 


NBC 


3.69 




FBC 


0.32 




BSGC 


3.01 


OTTO 





OTOTO 





8-PAM Hierarchical 





8-PAM 


BRGC/FBC 


1.18 




NBC 







BSGC 


oo 


16-PAM 


BRGC/FBC 


1.23 




NBC 







BSGC 


oo 



for reliable transmission at asymptotically low rates was found to take values between the SL 
— 1.59 dB and infinity. The asymptotic analysis was used to compare binary labelings for PAM 
and PSK input alphabets, as well as to predict the actual system performance at low rates when 
capacity-approaching codes are used. The asymptotically best labelings for M-PAM and A/-PSK 
with uniform input distributions appear to be the NBC and FBC, respectively. 

Using the first-order asymptotic of the BICM capacity, we analyzed the problem of FOO 
constellations for BICM. We showed that, under some mild conditions, the fading does not change 
the analysis of FOO constellations made for the AWGN channel. Interpreting the codewords of 
a binary labeling as the vertices of a hypercube, a constellation for BICM with uniform input 
distributions is FOO if and only if the input alphabet forms a linear projection of this hypercube. 
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Important special cases of this result are that constellations based on equally spaced M-PAM 
and M -QAM input alphabets are FOO if and only if the NBC is used. Another particular case 
are the hierarchical (nonequally spaced) Af-PAM input alphabets labeled by the NBC. We also 
showed that constellations based on constant-energy M -PSK input alphabets can never be FOO 
if M > 4, regardless of the binary labeling. 

In this paper, we focused on asymptotically low rates, and we answered the question about 
FOO constellations for this case. The analysis of second-order optimal constellations for BICM, 
and the dual problem for asymptotically high rates, or more generally, for any rate, is still an 
open research problem. 

Appendix A 
Proof of Theorem [7] 

In ll73l Theorem 3], the model Y = HX + Z is considered, where if is a matrix. This 
theorem states that the AMI between X and Y when if is known at the receiver can be 
expressed as 

Ix(X;Y) = ^trace (e a [H cov (X) if T ]) +0(N^) (58) 

when N — > oo, if the two following conditions are fulfilled: 

• There exist finite constants c > and d > such that Ex[||X|| 4+cr ] < c. 

• There exists a constant v > such that the matrix ff satisfies Pr{ || if || > 5} < exp(— 5 U ) 
for all sufficiently large 5 > 0. 

Since we consider real-valued vectors only, we have replaced the Hermitian conjugates in ll73l 
by transpositions in (1581) . Moreover, If73l Theorem 3] requires Z, X, and HX to be "proper 
complex". Nevertheless, the results are still valid if the two conditions in the items above are 
fulfilled, as explained in ll73l Remark 6]. 

The first condition is fulfilled since x , . . . , xm~i are all finite, and therefore, Ex[||X|| d ] < oo 
for all d > 0. The second condition is fulfilled because H = diag (if) and because of the 
condition © imposed on if. Moreover, since if = diag (if) and if contains i.i.d. elements, 
Efj [H cov (X)H T ] = E H [H 2 ] cov (X). The use of the identity trace (cov (X)) = E X [||X|| 2 ]- 
||E X [X]|| 2 , the definition of E s , and the relation SNR = E#[# 2 ]f^ in @g), gives (|47|). 
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Appendix B 
Proof of Theorem [TO] 
Expanding the inner sum in (|48T) . we obtain 

Pc k {umx\c k =u[X}\\ 2 = PcM x * P x\c k =o(x t 
«e{o,i} «ex fc ,o 



+ Pc k (l 



E XiP X \C k =l(Xi 



Using the identity ||a|| 2 + ||b|| 2 = \\\a — b\\ 2 + Ma + b\\ 2 and (I3TT) . we obtain 



P Ck (u)\\E xlCk=u [X] 

ue{o,i} 



PcM J2 x i p x\o k =o(xi) - y/P Ck (l) E x i p x\c k =i(xi) 



+ 



P Ck (0) J2 x l P x \cMx l ) + JP Ck (l) J2 x iPx\c k =i(xi) 



«ex fe ,o 



; XiP X {Xi 



; XiPx(Xi) 



1 

+ 2 



7m)^: iPx(Xi)+ jmm^ x ' Px( 



CkK 1 ) iez kl 



In this expression, and based on the definition of q i>k in ©, we recognize the first term as the 
first term inside the outer sum in (|49l) , and the second term as the second term inside the outer 
sum in (|49l) . This used in (l48l) completes the proof. 



Appendix C 
Proof of Theorem dU 

Consider any FOO constellation [Xpam, Um], where the binary labeling is defined by q^k 
for k = 0, . . . , m — 1 and i = 0, . . . , M — 1. From Theorem [121 there exist real values t> fc for 



fc = 0, . . . , m — 1 such that 



771—1 

fc=0 



(59) 



for z = 0, . . . , M — 1. We wish to find all combinations of q i:k and v k that satisfy ( 1591) . 

We start by giving two properties of the column vector V = [v , v±, . . . , i> m -i] T that will be 
used later in the proof. 
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• Since all pairwise differences x, — Xj = Ylk^il^k ~ Qj,k)vk are even numbers, and since 
Qi,k ~ Qj,k € {0, ±2}, we conclude that V £ Z m . 

• Because of (1591 ), the sum ±v ±vi± - ■ -±w m -i, with all combinations of signs, generates all 
the elements in X P am- Since X P am is formed by M distinct elements, ±v ±ui ± • • ■ ±v m _i 
must yield M different values, and therefore, for fc = 0, . . . , m — 1 must all be distinct. 

Consider a given bit position I £ {0, . . . , m — 1} and define, for i = 0, . . . , M — 1, 

Sj = Xj mod 2i>; 

m—1 

k=0 
k^l 

■m—1 

= ^2 qi ' kVk + Vi mod 2vh ^ 60 ^ 

fc=0 

where in the last step we have used the identity (a ± b) mod 2b = (a + b) mod 26. Because 
%i is an odd integer, Sj £ {1, 3, . . . , 2^ — 1} for all i We will now study the vector § = 
[s , si, . . . , sjvf_i] T and in particular count how many times each odd integer occurs in this 
vector. We will do this in two ways, in order to determine which values vi can take on. 

• It follows from (l60l) that Sj is independent of q^i for all i. Thus, if two codewords c, and 
Cj differ only in bit /, then Sj = Sj. This proves that each value 1,3,..., 2^ — 1 occurs an 
even number of times in S. 

• Because X is a vector of odd integers in increasing order and § consists of the same 
elements counted modulo 2vi, § consists of identical segments [1,3,..., 2v\ — 1] T of length 
V[. If Vi divides M, then § contains a whole number of such segments and each value in 
{1, 3, . . . , 2vi — 1} occurs exactly M/v\ times in §. If on the other hand v\ does not divide 
M, then the first and the last segment are truncated. In this case, S includes some values 
\_M/vi\ times and other values [M/vi\ + 1 times, where |_-J denotes the integer part. 

Since either [M/vi\ or \ M/vi\ + 1 is odd, and each value must occur in § an even number of 
times, we conclude from these two properties that V[ must divide M. Furthermore, the number 
of occurrences Mjv\ must be even, so vi must divide M/2. 

In conclusion, v , . . . , v m -i must all divide M/2, and their absolute values must be all distinct. 
Since M/2 has only m divisors, which are 1,2,4,..., 2 m_1 , they must all appear in V, but they 
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can do so in any order and with any sign. If V = [—1, —2, —4, . . . , — 2 m_1 ] T , then (|59l) is fulfilled 
by the NBC N m (see Example [6]). Negating w fc for any k corresponds to inverting bit k of the 
NBC, whereas reordering the rows of V corresponds to permuting columns in N m . 

Appendix D 
Proof of Theorem [T51 

Let M' = 2 m ', M" = 2 m ", and M = 2 m = 2 m ' +m " . To prove the "if" part, we assume that 
there exist two FOO constellations [X', L/, Um') an d [X", L", Um"]. Then, from Theorem [T2l 



m'-l 

k=0 
m"-l 

k=0 



[=J2l>l i = 0,...,M'-l, (61) 



i = 0,...,M"-l. (62) 



QM"l+j,k 



We analyze the two-dimensional constellation f2 constructed as f2 = [X, L, Uju] = [X'(g>X", L'® 
L",Um'm"]- It follows from the definition of the operator <g> that for all I — 0, . . . , M' — 1, 
j = 0, . . . , M" - 1, and k = 0, . . . , m - 1, 

?i,fe-m') k = m',...,m-l 
We will now show that f2 is FOO by explicitly constructing a matrix V that satisfies Theorem 
[T2l To this end, we define the vectors 

K,0], fc = 0,...,m'-l 
with ujj. and that satisfy (|6"TI)-(|62~1). The vectors constructed in this manner have the property 
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that for allZ = 0, . . . , M' - 1 and j = 0, . . . , M" - 1, 

m—l m' — l m" — l 

lM"l+j,k v k = X^ lM"l+j,k v k + X^ QM"l+j,k+m' v k+r 
k=0 k=0 k=0 

m'-l m"-l 



=0 fc=0 
Tt'-l m"-l 



k=0 k=0 
= XM"l+j- 

Substituting M"l + j = i yields (1531 ), which shows that f2 is FOO. Finally, to show that the 
constellation [X'®X", IIc(IL'<g)IL"), Um'm"] is also FOO, it suffices to observe that synchronously 
permuting the columns of Q(L) and V T does not change the right-hand side of (|52|) . which 
completes the proof of the "if" part^. 

For the "only if" part, consider any two-dimensional FOO constellation [X, L, Um]. By The- 
orem [Til the elements of X fulfill (l53l) . which can be decomposed into scalar equalities as 

m— 1 

Xi,n = X ft,* u *.n> i = 0, . . . , Af - 1, n = 0, 1 (63) 

k=0 

where Xi = [x i)0 , x^x] for i — 0, . . . , M — 1 and = [i> fcj0 , ffc,i] for A; = 0, . . . , m — 1. We will 
use this decomposition to characterize the points with the largest coordinate value in one of the 
dimensions. Because q ik G { — 1, 1}, x i>n takes values in the set ±u 0n ± • • • ±v m _ lin . The largest 
of these values is 

? A I I I I i i 

x n — max Xi n — ^0,71 ~r ■ • ■ t \v m -x n \. 

i=0,...,M-l 

If Vk,n 7^ for all k = 0, ... ,m — 1, then the symbol Xi for which x« in = x n is unique. If 
Vk,n = for one value of k, then x ijn = x n for two values of i, and so on. Generalizing, there 
exist 2 a symbols for which x^ n = x n if and only if there are a zeros among w ,n., • • • , ^m-i,n- 
Analogous relations hold for the minimum of 

For the special case when X is obtained from two one-dimensional input alphabets X' and X" 
as X = X' ® X", the two-dimensional symbols are XM"i+j = for 1 = 0,..., M' — 1 and 

14 An intuitive explanation for this is that reordering the bits of all codewords does not change the constellation's performance. 
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j = 0, . . . , M" - 1. We will prove that there exist labelings L' and L" such that [X', L', U M >] 
and [X", L", Um"] are both FOO, and we will identify the set of all such labelings. We do this 
by analyzing v kyTl for n = and 1 separately, beginning with n — 0. There are M" symbols 
Xi having x i)0 = x\ for each 1 = 0,..., M' — 1. This holds in particular for = xq. From the 
result in the previous paragraph, there are therefore m" zeros among v 0i0 , . . . , i> m -i,o- We will 
first consider the special case when the zeros are iw,o, • • • , iW-i,o> i.e., when 

K,0, • • • , %-l,0, • • • , Vm-ifi] = [v ofi , . . . U m '_ 1)0 , 0, . . . , 0], (64) 



m nonzero elements JTi zeros 



and will later generalize the obtained results to an arbitrary location of the m zeros. 
Assuming that (1641) holds, x\ can, for all I = 0, . . . , M' — 1, be written as 



x[ = XM"l+j,0 
m—l 



QM"l+j,kVk,0 



k=0 
m'-l 

= ^2 lM"i+j,kVk,o (65) 

fc=0 

where the second line follows from (1631) and the third from (|64|). The relation holds for all 



j=0,...,M"-l. 

We will now conclude from (1651) that 



q M "i+j,k = q M %k, l = 0,...,M' -1, j = 0, . . . , M" — 1, fc = 0, . . . , m' - 1. (66) 

This can be seen as follows. The sequence qwi+jfii ■ ■ ■ > QM"i+j,m'-i can take on 2 m ' = M' 
different values, because each element is ±1. For given values of t^o, these sequences all 
yield different values of x\ in (1651) . because these values are, by assumption, all distinct. Thus 
the sequence qM"i+j,o, ■ ■ ■ , QM"i+j,m'-i is uniquely determined by x\ and v 0>0 , . . . , w m '-i,o- Since 
both x\ and v k ,o are independent of j, so is qu"i+j,k- Therefore qM"i+j,k = qM"i,k- 
From this conclusion, (1631 ) simplifies into 

m'—l 

x[ = qM"i,kVk,o, / = 0, . . . , M' — 1, 

fc=0 

which is a one-dimensional version of (|53l ). It is satisfied only if [X', L', Um'] is FOO, where 
the elements ^' fc of Q(L') are 



q'ik = qM»i,k, l = 0,...,M' -1, k = 0,...,m' -1. (67) 
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A similar analysis for n = 1 shows that [X", L", Um"] is also FOO and, furthermore, yields 
analogous expressions to (l66l) and (1671) as 

q M »i+j,k = qj,k, / = 0,...,M'-1, j = 0, . . . , M" — 1, k = m',...,m-l (68) 

q'l k = q j>k+rn ,, j = 0,...,M"-l, k = 0,...,m"-l (69) 

where g" fc are the elements of Q(L"). 

Together, ([66]), ([67]), ©, and (HU) show that for I = 0, . . . , M' - 1 and j = 0, . . . , M" - 1, 

{<7;' ifc , fc = 0, . . . , m' - 1 
Qj,k-m'i k = m',...,m-l 
or, equivalently, that Q(L) = Q(L') <g) Q(L"). To convert this relation into a relation between 
(unmodified) labeling matrices L, L', and L", we can apply © to conclude that L is a column- 
permuted version of L' g) L". 

To complete the proof, we need to consider the case when the m" zeros among u 0)0 , ■ ■ ■ , u m _ 1)0 
are not the last m" elements as in (l64b . To this end, we apply an arbitrary row permutation to the 
V matrix, whose first column is given by (l64l) . Permuting the m rows of V means permuting the 
m elements of (|64|) . which in turn means that the m" zeros are shifted into arbitrary locations. 
Furthermore, as was observed in the first part of this proof, a row permutation of V corresponds 
to a column permutation of Q(L), or, equivalently, a column permutation of L. We can therefore 
conclude that regardless of where the m" zeros are located, the labeling L must be a column- 
permuted version of L' ® L". 

Appendix E 
Proof of Theorem [n] 

If Vq, . . . ,v m -i are orthogonal, then v^vj = for k ^ I. The symbol energies ||ccj|| 2 , for 
% = 0, . . . , M — 1, can be calculated from (1531) as 

m—1 m—1 

II X i\\ 2 = ^2^2<lhkqi,lVkVj 

k=0 1=0 
m—1 

E2 II 112 
Qi,k\\ v k\\ 

k=0 
m—1 

k=0 
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which is independent of i. This completes the "if" part of the theorem. 
For the "only if" part, we make use of the identity 

8bc T = \\a + b + cf -\\a + b- c\\ 2 - \\a - b + c\\ 2 + \\a-b- cf, (70) 

which holds for any vectors a, b, and c. Let X be any FOO constant-energy input alphabet and 
let k and I be any pair of distinct integers < k, I < m — 1. Define 

m— 1 



a= v i' 



i=o 
MM} 

b = v k , and c = From (1531) , the four vectors a ± b ± c all belong to X. Thus, all four have 
the same energy and the right-hand side of (1701) is zero. Thus v k vj = bc T = 0. This holds for 
all pairs of distinct k and /, which completes the proof. 

Appendix F 
Proof of Theorem [20] 

For P = , the average symbol energy is given by E s = Af 3 ~ 1 , and that the constellation is 
zero mean, i.e., Ex[-^] 2 = 0. Therefore, the coefficient a^ 1 in (|48T) is 

BI 



^EE ^c^\X?. (71) 

s fc=0 «G{0,1} 

For the BRGC, E x \c k =u[X] = for k = 1, . . . , m - 1 and u e {0, 1}. For A; = we find that 

Co=« 




which used in (1711) gives the desired result. 
For the NBC, we note that 



Using the fact that 



m-l / t \ 2fe ^ / j \ 



27 3 V 1 m 2 ; 



k 

the result a^ 1 = log 2 e is obtained!^ 

15 A similar argument for the proof of the NBC has been used in (23 
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That aj] 1 = if L m = S m follows trivially because of the construction of the BSGC, 
i.e., E x \c k =u[X] = for k = 0, . . . , m - 1 and u e {0, 1}. 

For the FBC, finally, its symmetry results in the same condition as for the BRGC, i.e., Ex|<7 fc=u [^] 
for k = 1, . . . , m — 1 and w G {0, 1}. Moreover, since for k = the BRGC and the FBC are 
the same, the coefficient a^ 1 is also the same. 



Appendix G 
Proof of Theorem EH 

For PSK and any k, P Ck (0)E xlCk=0 [X] + P Ck {l)E X \c k =i[X] = E X [X] = 0. Furthermore, 
since P Ck (0) = P Ck (l) = 1/2, ||E X | Cfc=0 [X] || 2 = ||E X | Cfc=1 [X]|| 2 . From these equalities, dH 
reduces to 



m— 1 



a 



131 



!og 2 e^ ||E X | Cfc=0 [X] 



fc=0 



4 log 2 e 

M 2 



m—l 



k=0 



(72) 



A. Proof for the BRGC 

Because of the symmetry of PSK input alphabets and the BRGC, || ^2 ieT x i\\ 2 m (1721 is 
zero for k = 2, ...,m — 1. Moreover, by symmetry, || ^ igIo a3i|| 2 = ||$2teXi ^ll 2, Since 
2o,o = {0, • • • , M/2 — 1}, the coefficient in (1721 is given by 



a 



bi _ 8 lo §2 e 



M 2 



Af/2-l 

E 

i=0 



8 log 2 e 
M 2 



tM/1-l , , \ 2 /Af/2-1 , , \ 2 



i=0 



i=0 



M 



(73) 



Using OH eq. (1.341.3)] we find that the first sum in |73]) is zero, and from J7H eq. (1.341.1)] 
the second sum in (1731) is equal to 1/ sin(7r/M). This completes the first part of the proof. 



B. Proof for the NBC 

For the NBC, || J2iex k Xi \\ 2 m ^721) is zero for k = 1, . . . ,m — 1. Moreover, since the first 
column of N m is always equal to the first column of G m , it is clear that the coefficient for the 
NBC is half of the one for the BRGC. 
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C. Proof for the BSGC 

By construction, S m = G m for all the columns except the first one, and therefore, only two 
bit positions contribute in the outer sum in (1721) . i.e., k = and k — 1. From the proof for the 
BRGC, the contribution for k = 1 is known to be 

1 



sin 2 (it /M) 



(74) 



For k = 0, we need the index set (cf. Example [Q) 

X 0i0 = {0, 4, . . . , M/2 - 4} U {3, 7, ... , M/2 - 1} 

U {M/2 + 1, M/2 + 5, . . . , M — 3} U {M/2 + 2, M/2 + 6, . . . , M — 2} 



M/8-1 



|J {4fc, M/2 - 1 - 4Jfc, M/2 + 1 + 4fc, M - 2 - 4fc}. 



(75) 



fc=0 



This partitioning of Z 00 into four subsets will now be used to calculate 



*ex o 



E 



cos 



(2i + 1)tt 
M 



E 



sm 



(2i + 1)tt 
M 



(76) 



We split the sum over I ,o m the second term of (1761) into four sums, one for each subset in 
(1751) . which yields 



E 



sm 



(2i + 1)tt 
M 



M/8-1 

E 

fc=0 



(l + 8A;)7r (M — 1 — 8k)n 

sm h sm — 



M 



M 



(M + 3 + 8£;)7r (2M - 3 - 

+ sm — — + sm v 



M/8-1 

E ( 2si 



k=0 



Applying UM eq. (1.341.3)] twice yields 

^ (2z + 1)tt 

2^ sm 



M 



M 

l + 8A;)7r 
M 

3tt 



— 2 sin 



M 

(3 + 8fc)vr 
M^ 



(77) 



Air 

2 cos — — 2 cos — esc — 



7T 



M 



7T 27T 

-2 sin — sec— , 
M M 



M 



M 



(78) 



where esc x = 1 / sin x is the cosecant function and sec x = 1/ sin x is the secant. 

Expanding the first term of (|76|) by the same method as in (1771) reveals that this term is zero. 
Now the result follows from CZI]), CZlK £Z6]), and d78l) . 
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D. Proof for the FBC 

By construction, the first bit of the FBC is the same as for the BRGC and the other bits are 
symmetric around M/2. Therefore, the components in the second dimension of || z~2iei k Xi \\ 2 
are zero for k = 1, . . . , M — 1 and (1721) can be expressed as 



fir 



„bi _ 41og 2 e 
M 2 

41og 2 e 
M 2 



m— 1 



sin 2 (7r/M) 



1 

sin 2 (7r/M) 



EE 

k=i \iex k ,o 

m-l / 

<E E 



cos 



(2i + 1)tt 
M 



cos 



(2i + 1)tt 
M 



(79) 



where X^ = {ie X M : i < M/2}. 
The indexes X^ of the FBC for k 



m — 1 are obtained as the indexes of the NBC 
of order m — 1. For example, for M = 32, we obtain X^ = {0,1,2,3,4,5,6,7}, X^q = 
{0, 1, 2, 3, 8, 9, 10, 11}, X 3 U = {0, 1, 4, 5, 8, 9, 12, 13}, and X£ = {0, 2, 4, 6, 8, 10, 12, 14}. This 
regularity results in a simplified expression of the inner sum in d79l) . i.e., 

2^ — 1_^ <2">Ti — k — \ -j^ 

£ C0S Pi+i>£ = £ E'ccfip-^ + a + irt (80) 



tan(7r/2 fc+1 ) 



(81) 



2sin(7r/M) ' 

where the final result was obtained by using If74l eq. (1.341.3)] twice in (|80l) , after some algebraic 
manipulation. Using (I8TI) in d79l) gives the desired result. 
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